Evaluation of Federated Learning Variations for COVID-19 diagnosis using Chest Radiographs from 42 US and European hospitals.

OBJECTIVE: Federated learning (FL) allows multiple distributed data holders to collaboratively learn a shared model without data sharing. However, individual health system data are heterogeneous. "Personalized" FL variations have been developed to counter data heterogeneity, but few have been evaluated using real-world healthcare data. The purpose of this study is to investigate the performance of a single-site versus a 3-client federated model using a previously described COVID-19 diagnostic model. Additionally, to investigate the effect of system heterogeneity, we evaluate the performance of 4 FL variations. MATERIALS AND METHODS: We leverage a FL healthcare collaborative including data from 5 international healthcare systems (US and Europe) encompassing 42 hospitals. We implemented a COVID-19 computer vision diagnosis system using the FedAvg algorithm implemented on Clara Train SDK 4.0. To study the effect of data heterogeneity, training data was pooled from 3 systems locally and federation was simulated. We compared a centralized/pooled model, versus FedAvg, and 3 personalized FL variations (FedProx, FedBN, FedAMP). RESULTS: We observed comparable model performance with respect to internal validation (local model: AUROC 0.94 vs FedAvg: 0.95, p = 0.5) and improved model generalizability with the FedAvg model (p < 0.05). When investigating the effects of model heterogeneity, we observed poor performance with FedAvg on internal validation as compared to personalized FL algorithms. FedAvg did have improved generalizability compared to personalized FL algorithms. On average, FedBN had the best rank performance on internal and external validation. CONCLUSION: FedAvg can significantly improve the generalization of the model compared to other personalization FL algorithms; however, at the cost of poor internal validity. Personalized FL may offer an opportunity to develop both internal and externally validated algorithms.

Use of Acute Mental Health Care in U.S. Children's Hospitals Before and After Statewide COVID-19 School Closure Orders.

Objective: This study aimed to examine changes in child emergency department (ED) discharges and hospitalizations for primary general medical (GM) and primary psychiatric disorders; prevalence of psychiatric disorders among acute care encounters; and change in acute mental health (MH) care encounters by disorder type and, within these categories, by child sociodemographic characteristics before and after statewide COVID-19­related school closure orders. Methods: This retrospective, cross-sectional cohort study used the Pediatric Health Information System database to assess percent changes in ED discharges and hospitalizations (N=2,658,474 total encounters) among children ages 3­17 years in 44 U.S. children's hospitals in 2020 compared with 2019, by using matched data for 36- and 12-calendar-week intervals. Results: Decline in MH ED discharges accounted for about half of the decline in ED discharges and hospitalizations for primary GM disorders (−24.8% vs. −49.1%), and MH hospitalizations declined 3.4 times less (−8.0% vs. −26.8%) in 2020. Suicide attempt or self-injury and depressive disorders accounted for >50% of acute MH care encounters before and after the statewide school closures. The increase in both ED discharges and hospitalizations for suicide attempt or self-injury was 5.1 percentage points (p<0.001). By fall 2020, MH hospitalizations for suicide attempt or self-injury rose by 41.7%, with a 43.8% and 49.2% rise among adolescents and girls, respectively. Conclusions: Suicide or self-injury and depressive disorders drove acute MH care encounters in 44 U.S. children's hospitals after COVID-19­related school closures. Research is needed to identify continuing risk indicators (e.g., sociodemographic characteristics, psychiatric disorder types, and social determinants of health) of acute child MH care.

Evaluation of electronic recruitment efforts of primary care providers as research subjects during the COVID-19 pandemic.

BACKGROUND: Recruiting healthcare providers as research subjects often rely on in-person recruitment strategies. Little is known about recruiting provider participants via electronic recruitment methods. In this study, conducted during the COVID-19 pandemic, we describe and evaluate a primarily electronic approach to recruiting primary care providers (PCPs) as subjects in a pragmatic randomized controlled trial (RCT) of a decision support intervention. METHODS: We adapted an existing framework for healthcare provider research recruitment, employing an electronic consent form and a mix of brief synchronous video presentations, email, and phone calls to recruit PCPs into the RCT. To evaluate the success of each electronic strategy, we estimated the number of consented PCPs associated with each strategy, the number of days to recruit each PCP and recruitment costs. RESULTS: We recruited 45 of 63 eligible PCPs practicing at ten primary care clinic locations over 55 days. On average, it took 17 business days to recruit a PCP (range 0-48) and required three attempts (range 1-7). Email communication from the clinic leaders led to the most successful recruitments, followed by brief synchronous video presentations at regularly scheduled clinic meetings. We spent approximately $89 per recruited PCP. We faced challenges of low email responsiveness and limited opportunities to forge relationships. CONCLUSION: PCPs can be efficiently recruited at low costs as research subjects using primarily electronic communications, even during a time of high workload and stress. Electronic peer leader outreach and synchronous video presentations may be particularly useful recruitment strategies. TRIAL REGISTRATION: ClinicalTrials.gov , NCT04295135 . Registered 04 March 2020.

Understanding Health Care Administrators' Data and Information Needs for Decision Making during the COVID-19 Pandemic: A Qualitative Study at an Academic Health System.

Objective. The COVID-19 pandemic created an unprecedented strain on the health care system, and administrators had to make many critical decisions to respond appropriately. This study sought to understand how health care administrators used data and information for decision making during the first 6 mo of the COVID-19 pandemic. Materials and Methods. We conducted semistructured interviews with administrators across University of Florida (UF) Health. We performed an inductive thematic analysis of the transcripts. Results. Four themes emerged from the interviews: 1) common types of health systems or hospital operations data; 2) public health and other external data sources; 3) data interaction, integration, and exchange; and 4) novelty and evolution in data, information, or tools used over time. Participants illustrated the organizational, public health, and regional information they considered essential (e.g., hospital census, community positivity rate, etc.). Participants named specific challenges they faced due to data quality and timeliness. Participants elaborated on the necessity of data integration, validation, and coordination across different boundaries (e.g., different hospital systems in the same metro areas, public health agencies at the local, state, and federal level, etc.). Participants indicated that even within the first 6 mo of the COVID-19 pandemic, the data and tools used for making critical decisions changed. Discussion. While existing medical informatics infrastructure can facilitate decision making in pandemic response, data may not always be readily available in a usable format. Interoperable infrastructure and data standardization across multiple health systems would help provide more reliable and timely information for decision making. Conclusion. Our findings contribute to future discussions of improving data infrastructure and developing harmonized data standards needed to facilitate critical decisions at multiple health care system levels. Highlights: The study revealed common health systems or hospital operations data and information used in decision making during the first 6 mo of the COVID-19 pandemic.Participants described commonly used internal data sources, such as resource and financial reports and dashboards, and external data sources, such as federal, state, and local public health data.Participants described challenges including poor timeliness and limited local relevance of external data as well as poor integration of data sources within and across organizational boundaries.Results suggest the need for continued integration and standardization of health data to support health care administrative decision making during pandemics or other emergencies.

Unraveling COVID-19: A Large-Scale Characterization of 4.5 Million COVID-19 Cases Using CHARYBDIS.

Purpose: Routinely collected real world data (RWD) have great utility in aiding the novel coronavirus disease (COVID-19) pandemic response. Here we present the international Observational Health Data Sciences and Informatics (OHDSI) Characterizing Health Associated Risks and Your Baseline Disease In SARS-COV-2 (CHARYBDIS) framework for standardisation and analysis of COVID-19 RWD. Patients and Methods: We conducted a descriptive retrospective database study using a federated network of data partners in the United States, Europe (the Netherlands, Spain, the UK, Germany, France and Italy) and Asia (South Korea and China). The study protocol and analytical package were released on 11th June 2020 and are iteratively updated via GitHub. We identified three non-mutually exclusive cohorts of 4,537,153 individuals with a clinical COVID-19 diagnosis or positive test, 886,193 hospitalized with COVID-19, and 113,627 hospitalized with COVID-19 requiring intensive services. Results: We aggregated over 22,000 unique characteristics describing patients with COVID-19. All comorbidities, symptoms, medications, and outcomes are described by cohort in aggregate counts and are readily available online. Globally, we observed similarities in the USA and Europe: more women diagnosed than men but more men hospitalized than women, most diagnosed cases between 25 and 60 years of age versus most hospitalized cases between 60 and 80 years of age. South Korea differed with more women than men hospitalized. Common comorbidities included type 2 diabetes, hypertension, chronic kidney disease and heart disease. Common presenting symptoms were dyspnea, cough and fever. Symptom data availability was more common in hospitalized cohorts than diagnosed. Conclusion: We constructed a global, multi-centre view to describe trends in COVID-19 progression, management and evolution over time. By characterising baseline variability in patients and geography, our work provides critical context that may otherwise be misconstrued as data quality issues. This is important as we perform studies on adverse events of special interest in COVID-19 vaccine surveillance.

Mental Health-Related Emergency Department Visits Among Children During The Early COVID-19 Pandemic.

Objective: To measure univariate and covariate-adjusted trends in children's mental health-related emergency department (MH-ED) use across geographically diverse areas of the U.S. during the first wave of the Coronavirus-2019 (COVID-19) pandemic. Method: This is a retrospective, cross-sectional cohort study using electronic health records from four academic health systems, comparing percent volume change and adjusted risk of child MH-ED visits among children aged 3-17 years, matched on 36-week (3/18/19-11/25/19 vs. 3/16/20-11/22/20) and 12-week seasonal time intervals. Adjusted incidence rate ratios (IRR) were calculated using multivariate Poisson regression. Results: Visits declined during spring-fall 2020 (n = 3892 vs. n = 5228, -25.5%) and during spring (n = 1051 vs. n = 1839, -42.8%), summer (n = 1430 vs. n = 1469, -2.6%), and fall (n = 1411 vs. n = 1920, -26.5%), compared with 2019. There were greater declines among males (28.2% vs. females -22.9%), children 6-12-year (-28.6% vs. -25.9% for 3-5 years and -22.9% for 13-17 years), and Black children (-34.8% vs. -17.7% to -24.9%). Visits also declined for developmental disorders (-17.0%) and childhood-onset disorders (e.g., attention deficit and hyperactivity disorders; -18.0%). During summer-fall 2020, suicide-related visits rose (summer +29.8%, fall +20.4%), but were not significantly elevated from 2019 when controlling for demographic shifts. In contrast, MH-ED use during spring-fall 2020 was significantly reduced for intellectual disabilities (IRR 0.62 [95% CI 0.47-0.86]), developmental disorders (IRR 0.71 [0.54-0.92]), and childhood-onset disorders (IRR 0.74 [0.56-0.97]). Conclusions: The early pandemic brought overall declines in child MH-ED use alongside co-occurring demographic and diagnostic shifts. Children vulnerable to missed detection during instructional disruptions experienced disproportionate declines, suggesting need for future longitudinal research in this population.

Secure Messaging and COVID-19: A Content Analysis of Patient-Clinician Communication During the Pandemic.

Background: Coronavirus disease 2019 (COVID-19) immediately impacted patient-clinician communication, particularly in the oncology setting. Relatedly, secure messaging (SM) usage greatly increased, yet it is unknown what was discussed and whether the technology was utilized to disseminate information. Aims: This study aimed at identifying the most frequently discussed topics using SM as well as at understanding how the communication process transpired during the early stages of the pandemic. Materials and Methods: A mixed-methods design was utilized, consisting of a content analysis of more than 4,200 secure messages, aggregated into 1,454 patient-clinician discussions. Data were collected from February 2020 to May 2020. Discussions were from various oncology departments and included physicians, physician assistants, and nurses. Based on the identified categories, a thematic analysis was conducted to understand the nuances occurring within discussions. Results: Out of the 1,454 discussions, 26% (n = 373) related to COVID-19. Of the COVID-19 discussion, the most frequently coded category was "changes, adjustments, and re-arranging care" (65%, n = 241), followed by "risk for COVID-19" (24%, n = 90), "precautions inside the hospital" (18%, n = 66), and "precautions outside the hospital" (14%, n = 52). Natural language processing techniques were used to confirm the validity of the results. Thematic analysis revealed that patients were proactive in rescheduling appointments, expressed anxiety about being immunocompromised, and clinicians were uncertain about providing recommendations related to COVID-19. Conclusions: The COVID-19 outbreak revealed the need for responsive and effective public health communication. The SM can disseminate information from trusted sources, clinicians, but can be better utilized to deliver tailored information for specific patient populations.

Research informatics and the COVID-19 pandemic: Challenges, innovations, lessons learned, and recommendations.

The recipients of NIH's Clinical and Translational Science Awards (CTSA) have worked for over a decade to build informatics infrastructure in support of clinical and translational research. This infrastructure has proved invaluable for supporting responses to the current COVID-19 pandemic through direct patient care, clinical decision support, training researchers and practitioners, as well as public health surveillance and clinical research to levels that could not have been accomplished without the years of ground-laying work by the CTSAs. In this paper, we provide a perspective on our COVID-19 work and present relevant results of a survey of CTSA sites to broaden our understanding of the key features of their informatics programs, the informatics-related challenges they have experienced under COVID-19, and some of the innovations and solutions they developed in response to the pandemic. Responses demonstrated increased reliance by healthcare providers and researchers on access to electronic health record (EHR) data, both for local needs and for sharing with other institutions and national consortia. The initial work of the CTSAs on data capture, standards, interchange, and sharing policies all contributed to solutions, best illustrated by the creation, in record time, of a national clinical data repository in the National COVID-19 Cohort Collaborative (N3C). The survey data support seven recommendations for areas of informatics and public health investment and further study to support clinical and translational research in the post-COVID-19 era.